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Abstract: E-Learning Sudan (ELS) is a custom-built computer/tablet game that provides alternative learning opportunities 
to Sudanese children who are excluded from education. Unique in ELS is that children can learn mathematics, in their own 
remote village, without a teacher. This research study assessed the effectiveness of ELS in two pilots through a pretest- 
posttest control group quasi-experimental design. In Pilot I, 67 children in three remote villages, aged used the game for a 
period of six weeks, five days a week, 45 minutes a day; the control group did not receive any education.. In Pilot II, 591 
children in 19 remote villages, played the game for six months, for a maximum of five times a week, 45 minutes a day; the 
control group received informal education in out-of-school centers. The results of the analysis on the pretest-posttest data 
revealed that ELS increased mathematics knowledge acquisition in numeracy and adding significantly and maintained 
student motivation to learn. Analyses of control group data and EGMA (internationally validates Early Grade Mathematics 
Assessment) showed that the children in the experimental group learned more than children who received no education at 
all, informal or formal education. These findings suggest that the implementation of ELS can greatly benefit learning for 
out-of-school children like in Sudan. 
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1 Introduction 

As other Sub-Saharan countries, Sudan struggles with issues regarding access to education and educational 
quality (Mtika 2010; The World Bank 2012). The push to achieve the goal of educating every child at least to 
primary level (enshrined in the Millennium Development Goals and now the Sustainable Development Goals) 
puts pressure on educational systems which already struggled to cope with demand. Although the push 
towards universal access has substantially increased education enrollment, reaching those without access to 
school is a pressing issue, particularly girls and children in the rural areas in Africa and South Asia (Kallaway 
2001; Cremin 2012). Equity still remains a major issue, not only involving girls, but also the poor, including 
linguistic and ethnic minorities, and other disadvantaged groups (The World Bank 2012; Kallaway 2001). 

There are approximately 2.3 million children in Sudan not in school. At present, it is not realistic to believe that 
this will be solved through traditional means (schools and teachers). Due to the cultural, geographic and socio¬ 
economic background of children, more flexible, empowering and affordable approaches outside formal 
schools are required. The focus of this approach should be on rural areas, communities affected by conflict - 
including Internally Displaced Persons (IDPs)- and specifically include girls and minority groups. Online and 
distance learning with ICT are seen as possible solutions. Digital technologies, if the software is well-designed 
and the content grounded in a well-constructed curriculum, can deliver one-to-one interactive instruction, in a 
consistent manner, to all children. Children can learn at their own pace and repeat material as often as they 
need. The use of technology also allows for the individual monitoring of progress. 

Promoting understanding of mathematics in the early years is critical, as longitudinal research has shown that 
early mathematical understanding is highly influential on later mathematics and reading performance at 
school (Duncan 2007), even after controlling for other basic skills that are known to affect school performance. 
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This study reports the results of two pilots in which a gamified version of the Sudanese curriculum for 
mathematics for out-of-school children was tested. The mathematics curriculum comprises the learning goals 
for Grades 1, 2, and 3 and leads to an official certification (Stubbe in press). 

Because this way of learning is completely different from the formal school system in Sudana proof of concept 
was needed before the three-year curriculum could be developed into a game. The most important question 
that needed to be answered was if children can learn mathematics by playing the game. Following the 'fastest 
route to failure', a small part of the game (Pilot I: six weeks of the curriculum; Pilot II: six months of the 
curriculum) was developed and tested in remote communities in three states in Sudan . 

Various meta-reviews and meta-analyses have shown the cognitive and motivational effects of serious games 
in general. In their meta-analysis of 32 studies in which traditional classroom teaching to computer gaming or 
interactive simulation were compared, Vogel et al. (2006) found an overall positive effect of serious games: 
significant higher cognitive gains and a more positive attitude towards learning were observed in subjects 
using interactive simulations or games versus traditional teaching methods. This seems to be the case for boys 
as well as girls, although the low number of studies that reported statistics for males and females gives reason 
to consider these results with caution. All age groups showed significant positive results for the use of 
computer gaming or interactive simulation. The type of activity did not appear to be influential; neither did 
realism of the pictures in the game. Wouters et al. (2013) investigated whether serious games are more 
effective in terms of learning and more motivating than conventional instruction methods. In their meta¬ 
analysis of 39 studies they found that serious games were more effective in terms of learning and retention, 
but were not more motivating than conventional instruction methods. Learners in serious games learned more 
than those taught with conventional instruction methods when the game was supplemented with other 
instruction methods, when multiple training sessions were involved and when players worked in groups. 

Research on digital mathematics interventions has shown increased motivation (Rosas et al. 2003), more 
positive attitudes towards mathematics (Ke 2009), and a better mastery of mathematics (Praet et al 2014; 
Steenbergen-Hu et al. 2013; Li et al. 2010; Rasanen et al. 2009) for children in Kindergarten and primary 
education. All of these studies were conducted with European or North American children. Recently, Pitchford 
(2015) evaluated the effectiveness of a tablet intervention for mathematics in a school in Malawi. She 
concluded that tablet technology can effectively support early year mathematical skills in developing countries 
if the software is carefully designed to engage the child in the learning process and the content is grounded in 
a solid well-constructed curriculum appropriate for the child's development stage. 

Clark et al. (2008) argue that the choice of media does not influence learning or media. Differences in 
instructional design prevail over the method of delivery. Usually the difference between passive and active 
learning and the clear and concise instruction determines variation in learning outcomes. This means that in 
the use of educational games the design of instruction delivered by these games is of crucial importance. This 
is in line with Wouters (2009) who argues that the alignment of learning outcomes and game type, the 
alignment of game complexity and human cognitive processes, attention for cognitive and motivational 
processes, research on specific mitigating effects, like gender, on game effectiveness should be considered in 
game design. 

1.1 Acquisition of mathematical skills 

Research shows that children develop mathematical skills at different levels before beginning formal schooling 
(USAID 2009).Children across cultures seem to bring similar types of skills to school, but do so at different 
levels (Guberman 1996). In general, children from low-income backgrounds begin school with a more limited 
skill set than those from middle-income backgrounds. This is related to the environment in which children 
grow up that enables them to understand the world, master language and get insight in the basic knowledge 
needed for mathematics (Greenman 2011). Furthermore, the rate of acquisition of mathematical skills can be 
influenced by the opportunities children have in their communities (Guberman 1996). Flousehold tasks and 
chores can get in the way of developing these skills, but they can also enhance the acquisition of these skills 
because they provide meaningful learning opportunities (buying or selling vegetables). Once children begin 
formal education, they use this informal knowledge when completing new tasks (Baroody in Copely 1999; 
Ginsberg 1981). 
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Between the ages of 3 and 9, the construction of number knowledge develops in more or less the same ways 
(USAID 2009). Becoming efficient at mathematics requires the automatization of the subsequent stage, rather 
than repeating the earlier stages. Children need to free up cognitive capacity to be able to solve more complex 
problems (Pellegrino 1987). With continued practice, children become more confident in their computational 
and problem solving skills. This puts a significant emphasis on good early mathematics experiences for 
children. 

Across countries, curricular and conceptual goals show similar subjects (USAID 2009): 

■ developing an understanding of whole numbers, including concepts of correspondence, counting, 
cardinality, and comparison; 

■ representing, comparing, and ordering whole numbers, and joining and separating sets; 

■ developing understandings of addition and subtraction, and strategies for basic addition and subtraction 
facts, including whole-number relationships (e.g. tens and ones); and 

■ developing understanding of base-ten numeration system and place-value concepts, including fluency 
with multi-digit addition and subtraction. 

This means that a mathematics game aimed at the first three grades of primary school, should comprise at 
least these subjects. 

1.2 Game design 

The design of the game and its requirements are described in more detail in Stubbe et al. (in press), but this 
paragraph will provide a short summary. The most important constraint for the game design was that children 
will use the game in their own remote communities. This automatically implies that there will be no school or 
teachers. Instruction and feedback will therefore have to be incorporated into the game. Because this project 
focuses on vulnerable children with little learning support from parents or teachers, we assume them to have 
little informal knowledge. Besides, the opportunities to learn from everyday life situations in the communities 
are scarce. Because of this the approach for struggling learners is followed. One of the major issues in 
supporting struggling learners is to ensure that there is a strong basis to build on. This corresponds with the 
concept of mastery learning (Bloom 1985), where 'the students are helped to master each learning unit before 
proceeding to a more advanced learning task'. Furthermore, struggling learners need explicit instruction 
(Timmermans 2005; Milo 2003). Research (Bodovski 2007) shows that struggling learners show less 
engagement during instruction. If this engagement is increased, performance increases as well. A focus on 
'time on task' (Carroll 1963) could help to improve learning results; for all children can learn mathematics, but 
some need more time than others. This means that the game should provide active learning, motivate children 
to increase their time on task, and use individual, continuous assessment in order to provide each child with 
mini-games that matches the level of mastery. In addition, special attention will need to be given to specific 
context in which the children live, to avoid children using too much of their cognitive capacity for processing 
(narrative) information that is not directly related to the learning content (Wouters et al. 2013). 

Instruction is provided by short videos (1-2 minutes) per mathematical concept (Figure 1). In the videos, 
slightly older children, aged 14-15, explain the mathematical concept and show how to approach problems 
relating to this concept. This has a motivating effect, and helps the children to understand what they need to 
do. The language used is simple Modern Standard Arabic, which makes it possible for children to understand 
the instruction, and, therefore, understand the mathematical concept. Children must watch the video once, to 
introduce a new mathematical concept. Then, they can watch it as many times as they like. 



Figure 1: Screenshot of instruction video Figure 2: Screenshot of feedback in mini-game 
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Children receive immediate feedback on the answers they give (Figure 2). This helps them to know if they are 
on the right track. The testing underlying this feedback is also used as continuous assessment to determine 
progress in the game and make sure children always learn at their own level. In addition, feedback on progress 
is given in two different ways: the children receive stars when they have finished mini-games, and proceed to a 
next level when they have finished the previous one. The curriculum in the game starts with Kindergarten 
learning goals. In this way, all children are given the opportunity to build a strong basis before continuing to 
more complex mathematical concepts. Each mathematical concept can be practiced by a number of different 
mini-games. First of all, this enables active learning: children are actively engaged in solving mathematical 
problems. In addition, the diversity of games motivates them to spend more time on practicing a mathematical 
goal. They feel they are doing different games, while they are, in fact, practicing the same mathematical 
concept. Last but not least, much attention was given to make the game context specific. For this, children in 
the remote communities in Sudan were asked to draw their own environment: clothes, food, animals, plants, 
and family. These drawing were used as a basis for the graphic design. The overall narratives (helping other 
children to become e.g. a goat herder; a shop where product are bought and sold) fit in with their cultural 
background. Because children recognize the objects and the narrative, their cognitive capacity can be 
dedicated to learning the mathematical concepts. 

2 Method 

This study reports two pilots. Both pilots used a pretest-posttest control group quasi-experimental design. 
Participation. 

In the communities, all children in the relevant age group were invited to participate in the experiment. 
Parents were informed about the goal and the method. The community was involved in setting up the 
'learning centers' (sheds where the children gathered to learn). Children were assigned to either morning or 
afternoon learning sessions, according to their parents' wishes. In this way learning could fit in with their 
chores and household tasks. 

Learning Sessions and Hardware 

Because there were two learning sessions a day, the hardware could be shared. Each laptop or tablet was used 
by two different children. Consequently, hardware stayed in the learning center, locked away until the next 
session. 

Facilitator 

Each community had a facilitator. The facilitator encouraged the children to work with the mathematics game 
and helped with technical problems. The facilitator was not supposed to teach or explain the principles of 
mathematics. Facilitators were trained to take this role and to solve technical problems. During the week, they 
lived in the communities, in the weekends they could go home. 

Observer 

During Pilot I, observers attended the learning sessions twice a week, to observe if the facilitator followed 
instructions. In both pilots they tested the children with respect to their mathematical knowledge. 

Staggered approach 

A staggered approach was used to support the start in each community by the observers and supervisors. 
Moreover, the technical issues that arose in the first community could be solved before the other communities 
started. The control groups were tested later. 

Oral test 

Oral mathematics tests were used because all children were assumed to be illiterate (Pilot I: test A; Pilot II: test 
A & B). These tests were designed on the basis of the Early Grade Mathematics Assessment (EGMA, USAID 
2009), and consisted of 30 items each (maximum score was 60 points), covering the very basics of 
mathematics. Both tests tested oral counting, number identification, one-to-one correspondence, quantity 
discrimination, word problems, addition and writing down numbers. In test A the numbers ranged from 1 to 
10, in test the numbers ranged from 1 to 20. The same tests were used as a pretest and posttest. 

Test protocol 

As the children live in remote communities, it was assumed they had not been tested before in any formal 
way. Reports on the testing of children in developing countries mention that children are shy to answer any 
questions at all (Kanu 2013). A test protocol was designed, including child-friendly approaches. The observers 
were trained to use this. During testing, a supervisor was present to ensure that the testing was performed 
according to protocol. 


www.eiel.org 


139 


ISSN 1479-439X 



The Electronic Journal of e-Learning Volume 14 Issue 2 2016 


Ethics 

The ethics committee of the Ahfad University for Women in Khartoum has approved these pilot studies. In 
addition, agreements have been signed by White Nile State, North Kordofan State and Gedaref State and the 
participating communities. All facilitators signed a child safety protocol. Parents have signed consent forms for 
their children to take part in the experiment and to be photographed. All data are related to a child-specific 
number. This was done for privacy reasons, as well as for pragmatic reasons (Arabic names can be spelt in 
different ways in English). The communities in the control group in these pilots, will participate in a later phase 
of the project. 

2.1 Pilot I 

Participants were 67 children in three remote communities in Sudan (Mona, OmTifag, and Wad Almoshmer), 
aged between 7-11. See Figures 3 and 4 for percentages per gender and age. The control group consisted of 19 
children in a fourth remote community (OmOkaz). None of the children had been to school before. The 
experimental group used the laptop game for a period of six weeks, five days a week, 45 minutes a day, while 
supervised by a facilitator. The control group did not receive any education in the same period. In addition to 
the test results (test A, pretest and posttest), the following data were collected: attendance, motivation 
(observed by facilitator) and the logged data in the mathematics game. At the end of the pilot the children 
were tested by a nearby school, to assess their Grade level. 



Figure 3: Percentage of children per gender Figure 4: Percentage of children per age 

Three children were excluded from the experimental group; two because they had been to school before, one 
because he had been brought from another community and became homesick after only three days. Three 
children did not take the post-test, seven others dropped out during the pilot. The data from the remaining 54 
children was used for analysis. In the control group no children were excluded from analyses. 

There was one technical problem in the first week in the first community. Because the game depends heavily 
on audio and video, children needed earphones to be able to hear the instruction from their own laptop. This 
was arranged within a week. Furthermore, there was a software problem: the first four levels in the game 
were adequately developed. For the fifth level, the instruction video had not been included, which made it 
difficult for the children to understand what they needed to do in that level. 

Although unique numbers were used to ensure anonymity, some observers only wrote down the names of the 
children. Because of the many ways in which Arabic names can be written down in English, it was hard to 
match the collected data for nine children. In collaboration with the observers, the names were matched with 
the unique numbers. 

2.2 Pilot II 

Participants were 591 children in 19 remote communities in three states in Sudan (White Nile, North Kordofan 
and Gedaref), aged 7-9. The control group consisted of 325 children in 10 more remote communities in the 
three states. The experimental group used the tablet game for a period of approximately six months, for a 
maximum of five days a week, 45 minutes a day, while supervised by a facilitator. The children in the control 
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group were enrolled in informal education; they attended two mathematics lessons a day of 45 minutes each, 
taught by a teacher in out-of-school centers. In addition to the mathematics tests A and B, the internationally 
validated Early Grade Mathematics Assessment (EGMA) was taken by independent consultants, with a 
stratified sample of the children (210) in the experimental condition. Finally, logged data were collected from 
532 accounts. 

Figures 5 and 6, below, show the percentages per gender and age. 



Three children were excluded from the experimental group; one because he was too young (6 years old), two 
because they had been to school before. Facilitators reported that 57 children dropped out during the pilot. 
Five more children were excluded on the basis of the logged data: their logged data showed they had only 
played the game for a short period, and not finished it. The data of the remaining 526 children was used in 
further analyses. There were, however, some issues with the data collection in the control group: the pretest 
and posttest were not taken at the designated times. In Gedaref, the pretest of test A was taken two months 
after the children had started their lessons. Because of this, all data from the Gedaref control group (100 
children) was excluded from further analysis. In White Nile and North Kordofan, the pretest of test A was taken 
at the right time; the posttests, however, were taken later than planned. Instead of an interval of 6-8 weeks, 
the posttest was taken after three months in North Kordofan (45 children) and after six months in White Nile 
(180 children). Consequently, the results from these two states were analyzed as two different sets. There 
were no posttest data for 14 children in North Kordofan and for 34 children in White Nile. The data of the 
remaining 177 children was included in further analyses. Because of logistic issues, logged data were not 
collected for two communities (57 children). Also, the matching of logged data to the test data proved difficult, 
because the facilitators had not used the unique child numbers for this, but had instead used the child's name. 
Because names in Arabic can be spelled in different ways, it was impossible to match 23 files of logged data. 
For 449 of the children in the experimental condition logged data were available and included in further 
analyses. 

Because of the iterative development process used, two updates of the game had to be installed during the 
pilot period. The progress in the communities in North Kordofan, who started first, was faster than anticipated: 
they had to wait two weeks before the first update could be installed. Furthermore, two mini-games did not 
function properly which made it impossible to give the right answers. These bugs were fixed within two weeks. 

In general, communities had two facilitators, taking turns supervising the learning sessions. In White 
Nile,however, there were four communities in which the only facilitator left in November 2014, and the new 
facilitator(s) started in January 2015, leaving a one to two-month interval during which learning sessions were 
not supervised. 

3 Results 
3.1 Pilot I 

Pilot I was conducted in the period of December 2012 to February 2013. 

There were more boys than girls, in the experimental condition (56%-44%) as well as in the control group 
(60%-40%). This reflected the situation in the communities; girls were not excluded from the pilot. There was 
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no significant difference in participation of girls and boys between the experimental communities and the 
control group. 

The children's ages varied between 7 and 11, with most children at the age of 8 and 9 There was no significant 
difference between the average age in the experimental condition and the control group (8.3). 

The experimental group had an average of 33% correct answers on the pretest (N=67, M=20.3, SD= 11.5). The 
control group had an average of 28% correct answers on the pretest (N=19, M=16.5, SD= 5.6). An independent 
T-test showed no significant differences on the scores of the pretest between the experimental group and the 
control group (t=-1.4283, df=79, p=0,16). The average score of the experimental condition on the post-test was 
55% correct (N=54, M=33.1, SD=15.5), the control group showed a slight increase to 29% correct answers 
(N=19, M=17.2, SD=5.3). An independent T-test showed a significant difference between the posttest scores of 
the experimental condition and the control group (t=-4.5059, df=79, p=0.00).Furthermore, an independent T- 
test showed significant differences on the delta scores (posttest-score minus pretest-score) between the 
experimental group and the control group (t=9.1, df=71, p=0.00) (see Figure 7). Hence, the experimental group 
has learned significantly more than the control group. 
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Figure 7: Results pretest-posttest 

All the children that used the game improved significantly, but children with a lower score on the pre-test, 
increased their scores more than children with a higher score on the pre-test. The increase varied between 9 
and 42 points. There were no significant differences between boys and girls or between age groups. 

3.1.1 Logged data 

The logged data show that with a few exceptions all the children used the game five days a week during the 
pilot period. Approximately 30% of the children completed all the materials in the game. The other 70% of the 
children finished between 50-70% of the materials. The children that had finished before the pilot ended were 
told to start again from the beginning. 

3.1.2 Observations 

Observation forms were filled out daily by the facilitators. They show most children were present during 
sessions and were motivated to learn most of the time (see figure 8). There were differences in motivation 
between communities at the start of the experiment, but average motivation increased over the six weeks 
(from 2.7 to 3.2 on a 4-point scale). In Mona motivation decreased during the first week. This could be 
explained by the fact that the children did not have headphones, and could not hear the instruction. After this 
was solved, motivation increased again. In Wad Almoshmer, motivation in the first week was at a maximum. 
Later, motivation decreased towards a more average level. 
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Motivation during experiment 
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Figure 8: Motivation during pilot i 

The children said they loved the instruction videos very much, and some watched them twenty times. When 
asked about this, they would say that the older children in the videos were their big brother or sisters (role 
model); 'If she can do it, I can do it as well!' They also loved the many games and the colors in the game. 

One of the participating children was a physically handicapped boy . Because of his slurred speech, people 
thought he was retarded as well. He performed very well with an average improvement of 15 points. He is now 
more accepted by the other children, and even his speech has improved. 

After the six-week experiment the children were taught by teachers in their own community for the rest of the 
school year. At the end of this period they were tested by a nearby school. Ten children were admitted to 
Grade 4, five children were admitted to Grade 2 (which would be normal progression) and the rest was 
admitted to Grade 3. The progress of the ten children admitted to Grade 4 was followed carefully. At the end 
of Grade 4 they were the top 10 of their year. This shows that the children not only learned their mathematics, 
but also formed a strong basis for further development. 

3.2 Pilot II 

Pilot II was conducted in the period of October 2014 to March 2015. Participation of boys and girls was almost 
equal (49% vs. 51%). The average age of the participating children was 7.8 years. Children in North Kordofan 
were slightly younger (average is 7.4 years), compared to the children in Gedaref (8.0 years ) and White Nile 
(7.9 years). Analysis of the demographic information of the children who dropped out showed no significant 
effect for gender or age. 

3.2.1 Test results 

To assess if children have increased their scores on test A an Anova repeated measures (SPSS GLM) test within 
subjects factor: Math-A-PRE en Math-A-POST was used. On average children in White Nile, North Kordofan and 
Gedaref had 33% correct on the pre-test of test A (20 of a max. of 60 points). The average on the post-test of 
test A was 68% correct (41 points). This increase is significant (F(l,499)=1170.929; p< .001); r=.85). There were 
significant differences between the states (F(2,99)=21.710; p < .001; r=.29); White Nile has a higher score than 
North Kordofan and Gedaref on the pre-test (24 points resp. 16 resp. 19) as well as on the post-test (47 points 
resp. 40, resp 37) of test A. Posthoc tests (Bonferroni) show that White Nile differs significant (p < .001) from 
the other two states. There is also a significant interaction between Math and State F(2,499)=9.055; p < .001; 
r=.21). All three states perform better on the Post-test than on the Pre-test, but North Kordofan has a relative 
greater increase of scores from pre-test to post-test. This can be explained by the significant differences 
between age groups, with older children having higher scores than younger children (F(2,499)=14.758; p < 
.001); r=.25) (see Figure 7). The average age in North Kordofan was lowest, in White Nile children are older. 
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Figure 7: Average score test A pre-post per age. 

Also, children with a lower score on the pre-test showed a larger increase in scores than children with a higher 
score on the pre-test (F=29.17, hypdf=2 errodf=488, p<.00, eta=.ll). This can partly be explained by a ceiling 
effect: if children have a higher score on the pre-test, there is less room for improvement. In addition, the first 
levels of the game addressed more basic mathematical concepts. If children already mastered these, they 
could not learn much in these levels. There were no significant differences for gender: scores as well as 
increase of scores were similar for boys and girls. 


Because of the data collection issues with the control group regarding the timing of testing, differences 
between the experimental group and the control groups must be interpreted with caution. Comparisons are 
made per state, not with the total average scores. A comparison of the North Kordofan (NK) experimental 
group with the NK control group shows no significant differences, in pre-test, post-test or increase of scores 
between pre- and posttest (see Figure 8). This is a positive finding, as the NK control group received twice as 
much instruction per day, compared to the experimental group (2 times 45 minutes, vs. 45 minutes a day). 
Moreover, the NK control group had a three month interval between pre- and posttest, whereas the 
experimental group had a 6-8 week interval between tests. Roughly, the NK control group has had three times 
as much opportunity to learn as the NK experimental group. The NK control group is very small (N=31), which 
makes it less suitable to compare to the NK experimental group (N=182). 


Average results test A, 
Pretest-Posttest 



Figure 8: Average results test A, Pretest-Posttest North Kordofan 


A comparison of the White Nile (WN) experimental group (N= 148) to the WN control group (N=146) shows 
that there is a significant difference in increase (31 vs 36 points) between the two groups (F(l,288)=17.034; p < 
.001; r=.24). The WN control group on average significantly increased its score more than the experimental 
group (see Figure 9). 
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Average results test A, 
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Figure 9: Average results test A, pretest-Posttest, White Nile 


It is important to note that the WN control group had (much) more opportunity to learn: they had two 
mathematics lessons of 45 minutes per day versus 45 minutes in the WN experimental group, and a much 
longer interval between pretest and posttest (6 months vs. 6-8 weeks). This means the WN control group had 
roughly six times more learning time than the children in the WN experimental group. The WN control group 
had 75% correct on the posttest, which means that there was more room for improvement. From this 
perspective, the difference in increase of scores is rather small. Children in the WN experimental condition 
have relatively learned more. 


To assess if children have increased their scores on mathematics test B an Anova repeated measures (SPSS 
GLM) test within subjects factor: Math-B-PRE en Math-B-POST was used. The average score of children in 
White Nile, North Kordofan and Gedaref on the pre-test of test B was 32 (max. 60). The average score on the 
post-test of test B was 41. This average increase of 9 points is significant (F(l,456)=160.067; p < .001; r=.51). 
There were no significant differences for gender (see Figure 10) and age groups. 


Average results test B, 


Pretest-Possttest 



B-Pre B-Post 


■ Boys 

■ Girls 


Figure 10: Average results test B, Pretest-Posttest for gender 

The pre-test score is remarkably high. This can only partly be explained by the overlap between test A and test 
B. In the research plan, the pre-test of test B should have been taken at the same time as the post-test of test 
A. In reality it was taken later. In the meantime children played on, and thus increased their knowledge. This 
may explain the high average on the pre-test of test B. 


Because the order of the curriculum in the game was different from the Sudanese curriculum used in the out- 
of-school learning centers, the control groups did not take test B: they had been taught the mathematical 
concepts tested in test B. 
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3.2.2 Results EGMA 


The results of EGMA, taken by 210 of the children in the experimental condition, can be compared to earlier 
studies with Arabic speaking children in Khartoum and Jordan, because it is an internationally validated test. 
Figure 11, below, shows that the children in the experimental condition (ELS) had the highest percentage 
correct in three sub-tests of EGMA (Shapes I, Shapes II and Word problems) after only six months of learning, 
compared to children who had attended school for 2.5 years in Khartoum and Jordan. In a fourth sub-test 
(Missing number) the children in the experimental condition had a higher score than the children from 
Khartoum. The children in the experimental condition only had a slightly lower score on: Number 
discrimination, Addition level 1, and subtraction level 1. 
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Figure 11: Scores EGMA compared; E-Learning children (ELS), Khartoum and Jordan 


There are significant positive correlations between the scores on the mathematics tests A and B (pre and post) 
and the measurements of EGMA. These correlations are not always strong. The strongest correlations are for 
Addition level 1, Subtraction level 1 and Problem solving, all correlating more than .50. Although all tests show 
significant positive correlations with EGMA, the B-Posttest has the strongest correlations. This can be 
explained by the fact that test B measures more difficult mathematical concepts than test A, and is, therefore, 
more similar to EGMA than test A. These correlations show that test A and B measure similar concepts of 
mathematics as EGMA. 


3.2.3 Logged data 

Most of the children played the game for a period of 5 to 7 months (average 135 days). Girls were found to 
participate for a longer period than boys (average of 141 versus 129 days; F(l,378) = 7,726; p < 0,05). No 
differences were found for age. Children from North Kordofan played a significantly shorter period (122 days), 
compared to White Nile (138 days) and Gedaref (145 days; F(2,377) = 11,114; p < .001). Correcting for both 
gender and age, children were found to only differ in the number of days between first and last play based on 
state (F (2,363) = 6.123; p<.05). 

On average, most children were found to participate two to three times a week. Figure 12 represents the 
frequency of playing (average plays per week), in categories. No differences were found for gender. Age was 
found to be significantly related to frequency of playing: 7-year olds played more often (2,7 days a week) than 
children 8-year olds (2,3 days a week), who in turn played more often than 9-year olds (2,0 days a week; 
F(2,377) = 17,909; p < .001). There was also a difference between states: children from North Kordofan played 
more often (3,2 days a week) than children from White Nile (2,2 days a week), who in turn played more often 
than children from Gedaref (1,7 days a week; F(2,377) = 143,954; p < .001). Correcting for both gender and 
age, children were found to only differ in the frequency of playing based on state (F (2,363) = 80.093; p < .001). 
Frequency of playing is an average that can be the result of playing five times a week for a number of weeks 
and then skipping one or more weeks, or playing a limited number of days per week. 
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Figure 12: Frequency of playing 

Although there is a correlation between the number of times played and percentage of the game completed, 
test results are better for children who participated for a longer period, even if their frequency of playing was 
lower. 

4 Discussion and conclusion 

In this study we aimed to test if children in rural areas in Sudan could learn mathematics autonomously, by 
playing a mathematics game on a computer or tablet, only supervised by a facilitator. To assess the 
effectiveness of the game two pilot studies were conducted. Pilot I tested a small part of the curriculum (6 
weeks) with 67 children playing the game five times a week, 45 minutes a day, supervised by a facilitator. 
Learning sessions took place in the remote communities. Pilot II tested a larger, and more diverse, part of the 
curriculum (6 months) with 591 children playing the game 45 minutes a day for a maximum of five days a 
week. In a baseline study the mathematical abilities of children were assessed. After approximately 6-8 weeks 
children were reassessed so that learning gains could be determined. In pilot II two tests were used, both with 
a 6-8 week interval. 

Educational research in remote areas in Sudan proved to be a challenge. Although facilitators and observers 
had been trained to use the test protocol, tests were not taken at the designated times. As a result the pre¬ 
test of test B was taken too late, leading to rather high scores on the pre-test. This makes it harder to show 
significant improvement. Using control groups in developing countries is also an issue: there is an ethical 
element in asking communities to participate in a pilot as a control group without allowing them to benefit 
from it. In addition, agreements have to be signed at various levels before communities can participate. As a 
result the control groups were smaller than the experimental group. Again, in the control group tests were not 
taken at the designated times, leading to an exclusion of the Gedaref control group, and modified analyses for 
the NK and WN control groups in Pilot II. 

Nevertheless, this study showed that using the mathematics game was more effective than no education, 
informal education and formal education (for the sub-tests taught in the game) for primary school children in 
Sudan. In both pilots, all children in the experimental condition have improved their scores on mathematics 
tests significantly. In pilot I, the control group who received no education did not improve their scores during 
the same period. This proves that the increase in mathematics scores in the experimental was not caused by 
maturation or a test effect - as a result of taking the same test within six weeks - but by playing the game. In 
Pilot II the control groups showed the same or slightly more improvement, despite the fact that they had 
received three to six times as much opportunity to learn than the children in the experimental condition. To 
ensure external credibility, independent, trained consultants took the standard Sudanese version of EGMA 
with a stratified sample of the experimental group in Pilot II. The results show that the children in the 
experimental condition, who had used the game for approximately six months, had higher scores on three sub¬ 
test of EGMA than children in Khartoum and Jordan who had been to school for 2.5 years. On a fourth sub-test 
the children in the experimental condition had a higher score than the children in Khartoum. This shows that 


www.eiel.org 


147 


ISSN 1479-439X 







The Electronic Journal of e-Learning Volume 14 Issue 2 2016 


children can learn from the mathematics game, with only a facilitator to supervise them. There was no 
significant difference between boys and girls; the game is as effective for girls as it is for boys. 

It is important to realize that the children involved in these pilots had never been to school before. In addition, 
80% of their parents have never finished primary school and are functionally illiterate. Although mathematics 
games have been tested in developed countries (Praet et al. 2013; Rasanen et al. 2009), and even in schools in 
developing countries (Pitchford 2015), this is the first time a mathematics game was tested with this specific 
target population: out-of-school children in remote areas without any access to school, teachers or learning 
materials. Representatives of the Ministry of Education in Sudan who have visited the communities during the 
pilots, and the consultants taking the EGMA test with the children expressed their surprise at how much the 
children were learning and how confident they were about their knowledge. 

In general, children with a lower score on the pre-test of test A improved more than children with a higher 
score on the pre-test. This is probably due to a ceiling effect: the game and the test focused on the very 
beginning of mathematics. Children that already had some knowledge of mathematics, achieved a higher score 
on the pretest and thus had less room for improvement. In addition, the game taught the basics of numeracy 
and addition. Children who knew how to do this could not learn very much from the game. They were the ones 
that went through the initial mini-games quickly. Others took longer to understand the basics of numeracy. 
Since the game offers children the possibility to learn at their own pace, they would always be playing mini¬ 
games that suited their level of mathematical skills. 

During Pilot I children played the game for six weeks, five times a week. Pilot II was organized similarly: 
facilitators provided learning sessions for six months, five times a week. However, the logged data showed that 
the majority of the children only played an average of two to three times a week, thus decreasing allowed 
learning time by almost half. Although the learning results would probably have been better if children had 
participated five times a week, progress was still good. This shows that the use of the mathematics game 
allows children to learn at their own pace: if they skip learning sessions for a day or a week at the time, they 
can always come back and continue where they left off. In a more formal classroom environment this would 
be very difficult; children would have missed instruction. 

When children learn autonomously, without a teacher to give instruction and feedback, and motivate them, 
there is always the issue of motivation. Will they stay motivated to keep on learning for a longer period of 
time? During Pilot I average motivation stayed high, and increased slightly. During Pilot II, focus group 
meetings were organized with children and parents. In discussions, both groups indicated that they liked 
learning and playing the game very much. 

The most important goal of this study was to prove that children can learn from playing the mathematics 
game. As this method of learning is completely different from what is generally used in Sudan, a 'proof of 
concept' was needed before the rest of the game was developed. Both pilot studies have shown that children 
can learn, autonomously, using the mathematics game. Tablet technology, including well-designed, 
curriculum-based, engaging software, could help reach the 2.3 million out-of-school children in Sudan and 
teach them the basics of mathematics. Although teachers play a very important role in children's learning and 
children should attend school, this game can provide learning for those who have no access to school (yet). 

Acknowledgements 

E-Learning Sudan is conceived through a collaboration between the Ministry of Education of Sudan, Ahfad 
University for Women in Khartoum and War Child Holland. It is funded through the Ministry of Foreign Affairs 
of The Netherlands, with additional funding from UNICEF Sudan. Curriculum and game development is 
provided by TNO. The game was produced by Flavour with support from creative partners in Sudan. 

References 

Bloom, B. S. (ed). (1985) Developing Talent in Young People. New York: Ballentine Books. 

Bodovski, K., and Farkas, G. (2007) "Mathematics Growth in Early Elementary School: The roles of Beginning Knowledge". 

Student Engagement and Instruction (p. 115-130). 

Carroll, J.B. (1963) "A model of school learning''. Teachers College Record, 64 , 723-733. 

Clark, R.E., Yates, K., Early, S. & Moulton, K. (2008). An analysis of the failure of Electronic Media and Discovery-based 
learning: Evidence for the performance benefits of Guided Training Methods. In Silber, K.H. & Foshay, R. (Eds.) 


www.ejel.org 


148 


©ACPIL 



Hester Stubbe et al 


Handbook of Training and Improving Workplace Performance, Volume 1: Instructional Design and Training Delivery. 
Somerset, NJ: Wiley 

Copely, J.V. (Ed.) (1999) Mathematics in the early years. Reston, Virginia: National Council of Teachers of Mathematics. 

Cremin, P. & Nakabugo, M. G. (2012) "Education, development and poverty reduction: A literature critique". International 
Journal of Educational Development 32 (2012) 499-506. 

Duncan, G.J., Claessens, A., Huston, A.C., Pagani, L.S., Engel, M., Sexton, H., Dowsett, C.J., Magnuson, K., Klebanov, P., 
Feinstein, L., Brooks-Gunn, J., Duckworth, K. & Japel, C. (2007) School Readiness and Later Achievement. 
Developmental Psychology 2007, Vol. 43, No. 6,1428-1446. 

Ginsberg, H.P., &Russel, R.L. (1981) "Social class and racial influences on early mathematical thinking". Monographs of the 
Society for Research in Child Development, 46(6), 1-69. 

Greenman, E., Bodovski, K., and Reed, K. (2011) "Neighborhood characteristics, parental practices and children's math 
achievements in elementary school". The Pennsylvania State University, PA, United States 

Guberman, S.R. (1996) "The Development of Everyday Mathematics in Brazilian Children with Limited Formal Education". 
Child Development, Volume 67, Issue 4, pages 1609-1623, August 1996. DOI: 10.1111/j.l467-8624.1996.tb01817.x 

Kallaway, P. (2001) "The need for attention to the issue of rural education". International Journal of Educational 
Development 21 (2001) 21-32. 

Kanu, A.S., Hengel, C. van (2013) A Report on the Baseline Assessment of Children's Early Literacy Abilities in Selected 
Primary Schools in Classes I, II, III. Woord en Daad, 2013. 

Ke, F. (2008) "A Case Study of Computer Gaming for Math: Engaged Learning from Gameplay?" Computers & Education 51 
(2008) 1609-1620. 

Li, Q. & Ma, X. (2010) "A Meta-Analysis of the Effects of Computer Technology on School Students' Mathematics Learning". 
Educational Psychology Review (2010) DOI: 10.1007/sl0648-010-9125-8 

Milo, B. (2003) Mathematics instruction for special-needs students. Thesis University of Leiden, Heerenveen: Brouwer & 
Wielsma. 

Mtika, P. & Gates, P. (2010) "Developing learner-centered education among secondary trainee teachers in Malawi: The 
dilemma of appropriation and application". International Journal of Educational Development 30 (2010) 396-404. 

Pellegrino, J.W., & Goldman, S.R. (1987). "Information processing and elementary mathematics". Journal of Learning 
Disabilities, 20, 23-32. 

Pitchford, N.J. (2015). "Development of early mathematical skills with a tablet intervention: a randomized control trial in 
Malawi". Frontiers in Psychology, April 2015, Vol 6, article 485. 

Praet, M. & Desoete, A. (2014). "Enhancing young children's arithmetic skills through non-intensive computerized 
kindergarten interventions: A randomized controlled study". Teaching and Teacher Education 39 (2014) 56-65. 

Rasanen. P., Salminen, J., Wilson, A.J., Aunio, P. & Dehaene, S. (2009). "Computer-assisted intervention for children with 
low numeracy skills". Cognitive Development (2009), D0l:10.1016/j.cogdev.2009.09.003. 

Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., Grau, V., Lagos, F., Lopez, X., Lopez, V., 

Rodriguez, P. & Salinas, M. (2003) "Beyond Nintendo: Design and Assessment of Educational Video Games for First 
and Second Grade Students". Computers & Education 40 (2003) 71-94. 

Steenbergen-Hu, S. & Cooper, H. (2013) "A Meta-Analysis of the Effectiveness of Intelligent Tutoring Systems on K-12 
Students' Mathematical Learning". Journal of Educational Psychology 2013, Vol. 105, No. 4, 970-987. 

Stubbe, H., Badri, A. Telford, R., Oosterbeek, S., van der Hulst, A. (in press) Formative evaluation of a mathematics game 
for out-of-school children in Sudan. 

Timmermans, R. (2005) Addition and subtraction strategies. Assessment and instruction, Nijmegen: Radboud University. 

The World Bank (2012) The status of the education sector in Sudan. African Human Development Series, 66608 UNESCO 

USAID (2009) Early Grade Mathematics Assessment (EGMA): A Conceptual Framework Based on Mathematics Skills 
Development in Children. EdData II Technical and Managerial Assistance, Task Number 2. 

Vogel, J.J., Vogel, D.S., Cannon-Bowers, J., Bowers, C.A., Muse, K. & Wright, M. (2013) "Computer Gaming and Interactive 
Simulations for Learning: A Meta-Analysis". University of Central Florida. 

Wouters, P., van Nimwegen, C., van Oostendorp, H. & van der Spek, E.D. (2013, February 4) "A meta-Analysis of the 
Cognitive and Motivational Effects of Serious Games". Journal of Educational Psychology. Advance online 
publication. DOI: 10.1037/a0031311. 

Wouters, P., van der Spek, E.D. & van Oosterndorp, H. (2009) Current Practices in Serious Game Research: A review from a 
Learning Outcomes Perspective. IGI Global. 


www.eiel.org 


149 


ISSN 1479-439X 



